Split Additive Manufacturing for Printed Neuromorphic Circuits

Printed and flexible electronics promises smart devices for application domains, such as smart fast moving consumer goods and medical wearables, which are generally untouchable by conventional rigid silicon technologies. This is due to their remarkable properties such as flexibility, non-toxic materials, and having low-cost per area. Combined with neuromorphic computing, printed neuromorphic circuits pose an attractive solution for these application domains. Particularly, the additive printing technologies can reduce large amount of fabrication complexities and costs. On the one hand, high-throughput additive printing processes, such as roll-to-roll printing, can reduce the per-device fabrication time and cost. On the other hand, jet-printing can provide point-of-use customization at the expense of lower fabrication throughput. In this work, we propose a machine learning based design framework, that respects the objective and physical constraints of split additive manufacturing for printed neuromorphic circuits. With the proposed framework, multiple printed neural networks are trained jointly with the aim to sensibly combine multiple fabrication techniques (e.g., roll-to-roll and jet-printing). This should lead to a cost-effective fabrication of multiple different printed neuromorphic circuits and achieve high fabrication throughput, lower cost, and point-of-use customization

Aging-Aware Training for Printed Neuromorphic Circuits

Printed electronics allow for ultra-low-cost circuit fabrication with unique properties such as flexibility, non-toxicity, and stretchability. Because of these advanced properties, there is a growing interest in adapting printed electronics for emerging areas such as fast-moving consumer goods and wearable technologies. In such domains, analog signal processing in or near the sensor is favorable. Printed neuromorphic circuits have been recently proposed as a solution to perform such analog processing natively. Additionally, their learning-based design process allows high efficiency of their optimization and enables them to mitigate the high process variations associated with low-cost printed processes. In this work, we propose a learning-based approach to address another major challenge of printed electronics, namely the aging of the printed components. This effect can significantly degrade the accuracy of printed neuromorphic circuits over time. For this, we develop a stochastic aging-model to describe the behavior of aged printed resistors and modify the training objective by considering the expected loss over the lifetime of the device. This approach ensures to provide acceptable accuracy over the device lifetime. Our experiments show that an overall 35.8\% improvement in terms of expected accuracy over the device lifetime can be achieved using the proposed learning approach

Co-Design of Approximate Multilayer Perceptron for Ultra-Resource Constrained Printed Circuits

Printed Electronics (PE) exhibits on-demand, extremely low-cost hardware due to its additive manufacturing process, enabling machine learning (ML) applications for domains that feature ultra-low cost, conformity, and non-toxicity requirements that silicon-based systems cannot deliver. Nevertheless, large feature sizes in PE prohibit the realization of complex printed ML circuits. In this work, we present, for the first time, an automated printed-aware software/hardware co-design framework that exploits approximate computing principles to enable ultra-resource constrained printed multilayer perceptrons (MLPs). Our evaluation demonstrates that, compared to the state-of-the-art baseline, our circuits feature on average 6x (5.7x) lower area (power) and less than 1% accuracy loss

Realization and training of an inverter-based printed neuromorphic computing system

Emerging applications in soft robotics, wearables, smart consumer products or IoT-devices benefit from soft materials, flexible substrates in conjunction with electronic functionality. Due to high production costs and conformity restrictions, rigid silicon technologies do not meet application requirements in these new domains. However, whenever signal processing becomes too comprehensive, silicon technology must be used for the high-performance computing unit. At the same time, designing everything in flexible or printed electronics using conventional digital logic is not feasible yet due to the limitations of printed technologies in terms of performance, power and integration density. We propose to rather use the strengths of neuromorphic computing architectures consisting in their homogeneous topologies, few building blocks and analog signal processing to be mapped to an inkjet-printed hardware architecture. It has remained a challenge to demonstrate non-linear elements besides weighted aggregation. We demonstrate in this work printed hardware building blocks such as inverter-based comprehensive weight representation and resistive crossbars as well as printed transistor-based activation functions. In addition, we present a learning algorithm developed to train the proposed printed NCS architecture based on specific requirements and constraints of the technology

Spatial-SpinDrop: Spatial Dropout-based Binary Bayesian Neural Network with Spintronics Implementation

Recently, machine learning systems have gained prominence in real-time, critical decision-making domains, such as autonomous driving and industrial automation. Their implementations should avoid overconfident predictions through uncertainty estimation. Bayesian Neural Networks (BayNNs) are principled methods for estimating predictive uncertainty. However, their computational costs and power consumption hinder their widespread deployment in edge AI. Utilizing Dropout as an approximation of the posterior distribution, binarizing the parameters of BayNNs, and further to that implementing them in spintronics-based computation-in-memory (CiM) hardware arrays provide can be a viable solution. However, designing hardware Dropout modules for convolutional neural network (CNN) topologies is challenging and expensive, as they may require numerous Dropout modules and need to use spatial information to drop certain elements. In this paper, we introduce MC-SpatialDropout, a spatial dropout-based approximate BayNNs with spintronics emerging devices. Our method utilizes the inherent stochasticity of spintronic devices for efficient implementation of the spatial dropout module compared to existing implementations. Furthermore, the number of dropout modules per network layer is reduced by a factor of $9\times$ and energy consumption by a factor of $94.11\times$, while still achieving comparable predictive performance and uncertainty estimates compared to related works

Highly-Bespoke Robust Printed Neuromorphic Circuits

With the rapid growth of the Internet of Things, smart fast-moving consumer products, and wearable devices, requirements such as flexibility, non-toxicity, and low cost are desperately required. However, these requirements are usually beyond the reach of conventional rigid silicon technologies. In this regard, printed electronics offers a promising alternative. Combined with neuromorphic computing, printed neuromorphic circuits offer not only the aforementioned properties, but also compensate for some of the weaknesses of printed electronics, such as manufacturing variations, low device count, and high latency. Generally, (printed) neuromorphic circuits express their functionality through printed resistor crossbars to emulate matrix multiplication, and nonlinear circuitry to express activation functions. The values of the former are usually learned, while the latter is designed beforehand and considered fixed in training for all tasks. The additive manufacturing feature of printed electronics allows the design of highly-bespoke designs. In the case of printed neuromorphic circuits, the circuit is optimized to a particular dataset. Moreover, we explore an approach to learn not only the values of the crossbar resistances, but also the parameterization of the nonlinear components for a bespoke implementation. While providing additional flexibility of the functionality to be expressed, this will also allow an increased robustness against printing variation. The experiments show that the accuracy and robustness of printed neuromorphic circuits can be improved by 26% and 75% respectively under 10% variation of circuit components

Improved Arithmetic Performance by Combining Stateful and Non‐Stateful Logic in Resistive Random Access Memory 1T–1R Crossbars

Computing-in-memory (CIM) is a promising approach for overcoming the memory-wall problem in conventional von-Neumann architectures. This is done by performing certain computation tasks directly in the storage subsystem without transferring data between storage and processing units. Stateful and non-stateful CIM concepts are recently attracting lots of interest, which are demonstrated as logical complete, energy efficient, and compatible with dense crossbar structures. However, sneak-path currents in passive resistive random access memory (RRAM) crossbars degrade the operation reliability and require the usage of active 1 Transistor–1 Resistance (1T-1R) bitcell designs. In this article, the arithmetic performance and reliability are investigated based on experimental measurements and variability-aware circuit simulations. Herein, it is aimed for the evaluation of logic operations specifically with fully integrated 1T–1R crossbar devices. Based on these operations, an N-bit full adder with optimized energy consumption and latency is demonstrated by combining stateful and non-stateful CIM logic styles with regard to the specific conditions in active 1T–1R RRAM crossbars